{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dxP3r4mvQRnX"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "uU6msr-hQZGj"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HI6jQZyLQouk"
      },
      "source": [
        "# Gemma - 3 Function calling.\n",
        "\n",
        "Google released Gemma-3 its new variant that comes with Multimodality, Multilingual support upto 140 global languages, 128K context window and also Agentic capabilites like structured responses and function calling. In this notebook, we will explore how to use Function calling with open source models from HuggingFace via Gemma 3 4B-it model\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/Gemma/[Gemma_3]Function_Calling_with_HF.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8zdBd9FqRNYi"
      },
      "source": [
        "## Setup\n",
        "\n",
        "### Select the Colab runtime\n",
        "To complete this tutorial, you'll need to have a Colab runtime with sufficient resources to run the Gemma model. In this case, you can use a T4 GPU:\n",
        "\n",
        "1. In the upper-right of the Colab window, select **▾ (Additional connection options)**.\n",
        "2. Select **Change runtime type**.\n",
        "3. Under **Hardware accelerator**, select **T4 GPU**.\n",
        "\n",
        "### Gemma-3 setup\n",
        "\n",
        "To complete this tutorial, you'll first need to complete the setup instructions at [Gemma setup](https://ai.google.dev/gemma/docs/setup). The Gemma setup instructions show you how to do the following:\n",
        "\n",
        "* Get access to Gemma on kaggle.com.\n",
        "* Select a Colab runtime with sufficient resources to run\n",
        "  the Gemma 3 4B-it model.\n",
        "* Generate and configure a Kaggle username and an API key as Colab secrets.\n",
        "\n",
        "After you've completed the Gemma setup, move on to the next section, where you'll set environment variables for your Colab environment.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UPG9Tt-xRXnN"
      },
      "source": [
        "### Install dependencies\n",
        "\n",
        "Run the cell below to install all the required dependencies.\n",
        "\n",
        "- Transformers: Latest version that supports Gemma3\n",
        "- Accelerate\n",
        "- DuckDuck Go Search - for function calling"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XpQ4yzBIHGbH"
      },
      "outputs": [],
      "source": [
        "!pip install git+https://github.com/huggingface/transformers.git"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hX82hPriIM3Z"
      },
      "outputs": [],
      "source": [
        "!pip install accelerate"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E8WujGTadPcA"
      },
      "outputs": [],
      "source": [
        "!pip install duckduckgo-search"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a_KJh4F1Rjav"
      },
      "source": [
        "# Gemma 3\n",
        "\n",
        "Google has released Gemma-3, a successor to its earlier Gemma models, focusing on open-source accessibility and multimodal vision capabilities. The model is designed to handle complex tasks with extended context lengths, support for multiple languages, and vision capabilities, making it suitable for a wide range of applications.\n",
        "\n",
        "Gemma-3 has gained attention for outperforming notable models like DeepSeek V3, o3-mini, and Llama 3 405B, with extended context lengths, multilingual support, and multimodal capabilities.\n",
        "\n",
        "- Extended context length from 32K to 128K using ROPE scaling (from 10K to 1M) on global self-attention layers.\n",
        "- Available in 1B, 4B, 12B, and 27B sizes (Vision capability included, except for 1B, with bi-directional attention).\n",
        "- Supports an extensive range of 140 global languages.\n",
        "- Additional Features: Supports function calling, where the model can generate code or instructions to call external functions, and structured outputs, enabling it to produce data in specific formats like JSON or tables."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xUSjniskUsyc"
      },
      "source": [
        "## Load Model weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "us9dCp87GgVN"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "from transformers import AutoProcessor, Gemma3ForConditionalGeneration"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0p-2edQBR4af"
      },
      "outputs": [],
      "source": [
        "model_id = \"google/gemma-3-4b-it\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LISsLCCiGUVm"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "83f33e2bb29544ddb2dfd8da62c9ca8d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Using a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.50, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.\n"
          ]
        }
      ],
      "source": [
        "model = Gemma3ForConditionalGeneration.from_pretrained(\n",
        "    model_id, device_map=\"auto\"\n",
        ").eval()\n",
        "\n",
        "processor = AutoProcessor.from_pretrained(model_id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bnzOHRPtoQut"
      },
      "outputs": [],
      "source": [
        "torch.cuda.empty_cache()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i47OYekFdM4V"
      },
      "source": [
        "## Function Calling Gemma\n",
        "\n",
        "Function calling is a feature that allows large language models to interact with external systems, APIs, and tools in a structured way. This feature enables LLMs to not just generate text responses but to invoke specific functions when appropriate based on user queries. At its core, function calling involves describing functions to the LLM and having it intelligently return the function to call along with necessary arguments for particular tasks. For example, an LLM can be configured to retrieve stock price data by calling a specific function when a user asks about the current price of a particular stock."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UgU2fCoycyhu"
      },
      "outputs": [],
      "source": [
        "from duckduckgo_search import DDGS\n",
        "\n",
        "def search(query:str):\n",
        "  \"\"\"\n",
        "  search results to the user query\n",
        "\n",
        "  Args:\n",
        "      query: user prompt to fetch search results\n",
        "  \"\"\"\n",
        "  req = DDGS()\n",
        "  response = req.text(query,max_results=4)\n",
        "  context = \"\"\n",
        "  for result in response:\n",
        "    context += result['body']\n",
        "  return context"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CQiUJlCBdb8Y"
      },
      "outputs": [],
      "source": [
        "tools = [search]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "R3z_9TbvufF3"
      },
      "outputs": [],
      "source": [
        "query = \"which teams played Cricket ICC Champions Trophy 2025 finals and who won?\" # this is a realtime query, this event occured on March 9th 2025"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jHQrvzvRR7n8"
      },
      "source": [
        "## Define prompt for function calling\n",
        "\n",
        "Function Calling follows these steps:\n",
        "\n",
        "- Your application sends a prompt to the LLM along with function definitions\n",
        "- The LLM analyzes the prompt and decides whether to respond directly or use defined functions\n",
        "- If using functions, the LLM generates structured arguments for the function call\n",
        "- Your application receives the function call details and executes the actual function\n",
        "- The function results are sent back to the LLM\n",
        "- The LLM provides a final response incorporating the function results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "j4LiCaOodjyn"
      },
      "outputs": [],
      "source": [
        "conversation = [\n",
        "    {\n",
        "        \"role\": \"system\",\n",
        "        \"content\": [{\"type\": \"text\", \"text\": \"\"\"\n",
        "        You are an expert search assistant. Use `search` to get the most accurate details.\n",
        "        At each turn, if you decide to invoke any of the function(s), it should be wrapped with ```tool_code```. The python methods described below are imported and available,\n",
        "        you can only use defined methods. The generated code should be readable and efficient. The response to a method will be wrapped in ```tool_output``` use it to call more tools or generate a helpful, friendly response.\n",
        "        When using a ```tool_call``` think step by step why and how it should be used.\n",
        "\n",
        "        The following Python methods are available:\n",
        "        \\`\\`\\`python\n",
        "        def search(query:str):\n",
        "          \"\n",
        "          search results to the user query\n",
        "\n",
        "          Args:\n",
        "             query: user prompt to fetch search results\n",
        "          \"\n",
        "        \\`\\`\\`\n",
        "\n",
        "        User: \\{user_message\\}\n",
        "        \"\"\"}]\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"user\",\n",
        "        \"content\": [\n",
        "            {\"type\": \"text\", \"text\": query}\n",
        "        ]\n",
        "    }\n",
        "]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nh51Dr6Xde9r"
      },
      "outputs": [],
      "source": [
        "inputs = processor.apply_chat_template(\n",
        "            conversation,\n",
        "            tools=tools,\n",
        "            add_generation_prompt=True,\n",
        "            return_dict=True,\n",
        "            tokenize=True,\n",
        "            return_tensors=\"pt\",\n",
        ").to(model.device, dtype=torch.bfloat16)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xNO25997ebXH"
      },
      "outputs": [],
      "source": [
        "outputs = model.generate(**inputs, max_new_tokens=512)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XN32I1J2ltjN"
      },
      "outputs": [],
      "source": [
        "output = processor.decode(outputs[0], skip_special_tokens=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "moOkqbxAo0TT"
      },
      "outputs": [],
      "source": [
        "response = output.split(\"model\")[-1]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bGF5oJyntu7C"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "```tool_code\n",
            "search(query=\"Which teams played in the Cricket ICC Champions Trophy 2025 final and who won?\")\n",
            "```\n"
          ]
        }
      ],
      "source": [
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pldp3z7USS5P"
      },
      "source": [
        "## Execute the function calling code"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ni57IK_OpIya"
      },
      "outputs": [],
      "source": [
        "def extract_tool_call(text):\n",
        "    import io\n",
        "    from contextlib import redirect_stdout\n",
        "\n",
        "    pattern = r\"```tool_code\\s*(.*?)\\s*```\"\n",
        "    match = re.search(pattern, text, re.DOTALL)\n",
        "    if match:\n",
        "        code = match.group(1).strip()\n",
        "        # Capture stdout in a string buffer\n",
        "        f = io.StringIO()\n",
        "        with redirect_stdout(f):\n",
        "            result = eval(code)\n",
        "        output = f.getvalue()\n",
        "        r = result if output == '' else output\n",
        "        return f'```tool_output\\n{r}\\n```'''\n",
        "    return None"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SRYVbLKdosaN"
      },
      "outputs": [],
      "source": [
        "keyword = extract_tool_call(response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eznPlrIBpddw"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"```tool_output\\nThe ninth edition of the Champions Trophy 2025 saw India being crowned as the winners on 9th March 2025 after they overcame New Zealand in the final. Several exceptional performers lit up the tournament with the bat and ball. The best of them made it to the Team of the Tournament. Here's what the side looks like: 1. Rachin Ravindra (New Zealand)The 2025 ICC Champions Trophy was the ninth edition of the ICC Champions Trophy, a quadrennial ODI cricket tournament organised by the International Cricket Council (ICC). In November 2021 as part of the 2024-2031 ICC men's hosts cycle, the ICC announced that the 2025 Champions Trophy would be played in Pakistan. [4]On 19 December 2024, following an agreement between Board of Control for ...Official ICC Champions Trophy, 2025 Cricket website - live matches, scores, news, highlights, commentary, rankings, videos and fixtures from the International Cricket Council. ... 2025. ICC announces Champions Trophy 2025 Team of the Tournament ... Kohli stars as India march into final | Champions Trophy 2025. 4 March, 2025. ICC Champions ...In a thrilling ICC Champions Trophy final at the Dubai International Cricket Stadium on March 9, 2025, India emerged victorious over New Zealand, clinching the title in a closely contested match ...\\n```\""
            ]
          },
          "execution_count": 177,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "keyword"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ek9wlj3xuOnc"
      },
      "outputs": [],
      "source": [
        "prompt = [{\n",
        "        \"role\": \"user\",\n",
        "        \"content\": [\n",
        "            {\"type\": \"text\", \"text\": f\"Results from the search:{keyword} User_query : {query}\"}\n",
        "        ]\n",
        "    }\n",
        "]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QB331y_TvN7K"
      },
      "outputs": [],
      "source": [
        "final_prompt = processor.apply_chat_template(\n",
        "            prompt,\n",
        "            tools=tools,\n",
        "            add_generation_prompt=True,\n",
        "            return_dict=True,\n",
        "            tokenize=True,\n",
        "            return_tensors=\"pt\",\n",
        ").to(model.device, dtype=torch.bfloat16)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rcjE05aSvHEI"
      },
      "outputs": [],
      "source": [
        "response = model.generate(**final_prompt, max_new_tokens=512)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aBcgbJjEvlW4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "user\n",
            "Results from the search:```tool_output\n",
            "The ninth edition of the Champions Trophy 2025 saw India being crowned as the winners on 9th March 2025 after they overcame New Zealand in the final. Several exceptional performers lit up the tournament with the bat and ball. The best of them made it to the Team of the Tournament. Here's what the side looks like: 1. Rachin Ravindra (New Zealand)The 2025 ICC Champions Trophy was the ninth edition of the ICC Champions Trophy, a quadrennial ODI cricket tournament organised by the International Cricket Council (ICC). In November 2021 as part of the 2024-2031 ICC men's hosts cycle, the ICC announced that the 2025 Champions Trophy would be played in Pakistan. [4]On 19 December 2024, following an agreement between Board of Control for ...Official ICC Champions Trophy, 2025 Cricket website - live matches, scores, news, highlights, commentary, rankings, videos and fixtures from the International Cricket Council. ... 2025. ICC announces Champions Trophy 2025 Team of the Tournament ... Kohli stars as India march into final | Champions Trophy 2025. 4 March, 2025. ICC Champions ...In a thrilling ICC Champions Trophy final at the Dubai International Cricket Stadium on March 9, 2025, India emerged victorious over New Zealand, clinching the title in a closely contested match ...\n",
            "``` User_query : which teams played Cricket ICC Champions Trophy 2025 finals and who won?\n",
            "model\n",
            "According to the search results:\n",
            "\n",
            "*   **Teams:** India and New Zealand\n",
            "*   **Winner:** India won the final on March 9, 2025.\n"
          ]
        }
      ],
      "source": [
        "print(processor.decode(response[0], skip_special_tokens=True))"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "[Gemma_3]Function_Calling_with_HF.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
